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Administrative Concerns in Developing 
Programs of Applied Performance Testing 



Can your students fill out a job application? Can they read and understand 
an editorial in a newspaper? How many can fill out an income tax form properly? 
Can they compare two or three different models of a refrigerator and decide 
which offers them the best quality based on what they can afford to pay? Can 
they decide how much rent they can afford based on an income of 175 dollars a 
week? Can they decide which can of peas will give them more for their money? 
Can they determine how much it would cost them to drive to a city 150 miles 
away?* 

Many citizens feel questions like these are important for students to under- 
stand if they are to survive in today's complex world. They also feel that pub- 
lic schools should be offering students the opportunities to acquire skills that 
will be of value in their everyday living. 

Once again, the challenge is offered to those in administration to respond 
to the demands of the times. As a matter of fact, how well we are able to an- 
swer these questions may be more important in the coming years than college board 
scores or how many students make the honor roll. They may even be more important 
than the won-loss record of the football team. 

Applied Performance Testing offers us a tool for examining these concerns » 
Applied Performance Testing (APT) is defined as tests designed to measure 

performance in an actual or simulated setting. In education, performance? tests 
focus on the measurement of performance of tasks significant to the student's 
life outside the school or to adult life J' 



*This presentation reflects an attempt to assimilate material drawn from 
two papers developed by this author for the Clearinghouse for Applied Performance 
Testing (see Bibliography). 3 
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This is an intriguing statement for us as superintendents, curriculum coor- 
dinators, principals, and department chairmen^ It is intriguing because it pro- 
vides a bridge which will help us connect school learning to that real and very 
complex world outside the classroom. It gives us an opportunity to demonstrate 
that we really are concerned with what is needed and vdiat is practical. It in- 
dicates that we are concerned not only with a select group of students but all 
that we are charged with educating. 

For the first time, we may be able to demonstrate that our students are 
wiser consumers, that they do know how to make their opinions known to congress- 
men and other legislators, that they are able to fill out job application forms, 
that they can read and react to magazine and newspaper articles, that they do 
have an understanding of certain skills that are unique to their community, that 
they do have a better understanding of basic health needs, and that, in general, 
they are better able to cope with adult living. 

The Genesis of Applied Performance- Testing. in Education 

Why is APT enjoying such popularity today? As is often the case in educa- 
tional innovations, the answer may be found in societal forces outside the school 
rather than in any internal demand from educators themselves. 

In the 1960 *s, the federal government attempted to reduce inequality in ed- 
ucation through substantial funding and compensatory programs. As Schneider 
indicates I "When many of these programs failed to make much difference, reformers 
shifted their attention from redistribution [of resources] to gaining more control 
over decision-making in school systems (p. 3) Thus, during the 1970 's there 
has been a tharust by both citizens and government to control the decision-making 
process associated with education by forcing schools to produce more information 
related to student performance. The emphasis on assessment and accountability is 
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reflectea in performance contracting, management by objectives, PPBS and con-- 
tract learning. Note that these accoimtability techniques are concerned with 
output variables rather than input variables involving number of students, qual- 
ifications of teachers and number of textbooks- Previously, school evaluations 
have been mostly concerned with input variables. 

Applied Performance Testing is a tool of accountability and as such poss- 
esses all of the potential political ramifications of being equated with quality 
education, survival education, teacher competency and efficient school management. 
Although it may serve as an output measure for all these concepts, it is identi- 
cal with none. It is certainly susceptible to political distortion, comiriunity 
resistance, inefficient management, economic constraints and overblown claims of 
grandeur. 

The validity of APT rests on the following assunqptions : 

1. By indicating specific standards of performance for each student in 

practical life skills, the overall quality of education will be improved. 

2> Specific performance measures will indicate to t^ducgiors and the public 

exactly what learnings have taken place. 

3. Applied Performance Testing will help make teachers and administrators 
more accountable for what they do with children- 

4. Applied Performance Testing will help to direct public education toward 
practical kinds of programs oriented to the needs of each individual and each 
community. 

5. Applied Performance Testing will equip each student with the skills 
necessary to survive in the adult world. 

6. Applied Performance Testing will help the school reexamine and more 
clearly define the goals and objectives of its instructional programs. 
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7. Applied Performance Testing will lead to greater commiinity involvement 
in the educational process. 

8. Applied Performance Testing will help to restore public contidence in 
schools. 

Although Number 3 above may make some educators feel uncomfortable, the as- 
sumptions would appear to be acceptable to most administrators. They also por- 
tend a number of significant changes in school operations that must be considered. 

How will APT modify school processes? 

From a sociological viewpoint, schools exist to fulfill four functions: 
instruction, socialization, custodialization and certification (Spady, B.]. Ap- 
plied Performance Testing may have an impact in all these areas and in their re- 
lationship to one another. 

Relative to instruction, certainly APT will bring about changes in curricu- 
lum concerning both the content offered and the way it is structured. Material 
will have to be highly individualized and directed toward task performance re- 
lated to the needs of each community. Instruction will become less private as 
teachers are asked to interact with one another and with those citizens in the 
community who can offer facilities and expertise relative to testing needs. 
Thus, the teacher may play a less powerful role in determining content, modes of 
instruction and evaluation. Instruction will be offered at a number of sites 
both in the school and in the community. Work arrangements will be restructured 
and roles redefined. 

Applied Performance Testing may tie the socializing process of the school 
closer to the norms, attitudes and values of local communities. This closer re- 
lationship will present a philosophic and psychological base on which curri- 
culum content will be structured. The school will be more vulnerable and, by 
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necessity, more receptive to its immediate environment . For the first time, many 
citizens hold positions of low influence in the community may be asked to take 
an active part in the educational process. For instance, the man who runs the 
local lawn mower repair business in his garage may be asked to help develop a per- 
formance test concerning the tune-up of a lawn mower. 

The certification processes now in evidence ivill be greatly modified. While 
the Cavnegie xxnit will probably remain as the primary unit of evaluation, spe- 
cific competency scores and checklists will also be used. Furthermore, students 
who cannot qualify for the traditional diploma may be issued certificates of 
cosqpetency indicating specific ar^as in which skills have been exhibited. 

School evaluation tasks will have to be modified as follow: 

1. Develop and provide on demand equivalency examinations for all 

required courses whereby full course credit may be given. 

2. Develop and provide opportunity for competency in specific 

skill areas. 

3. Develop and administer criterion-based evaluation schemes for 

each skill area. 

4. Credit alternative learning experiences. 

5. Modify attendance and time requirements (Schneider, p. 2). 
Therefore, in a fully-developed APT model, the school will have to develop 

equivalency examinations for all specified skill areas as well as alternate forms 
of each exam and additional diagnostic materials. Students who cannot meet the 
competencies will have to be given extra help and attention. Credit will be 
awarded for alternate learning experiences particularly those involving on-the- 
job training. Also attendance and the 12^year custodial period will be modified-* 
some students may be able to complete all tasks in ten or eleven years whereas 
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others may have to remain in school for longer periods of time- Marks will not 
be based only on "attitude, punctuality, attendance, deportment.*' (Spady, AO 
Arrangements will have to be made so that performance indicators will be trans- 
ferable from one school to another « 

Schools will have to become more sophisticated in planning techniques. 
They will have to be able to react much more quickly to be capable of utilizing 
new technological developments for educational purposes. The school system will 
have to be much more adept at searching out and using resources and expertise 
that exist in society. The school will have to develop the capability to expose 
learners to a wide variety of realistic learning experiences. 

Potential Problems related to Applied Performance Testing 

Applied Performance Testing is fraught with potential problems as well as 
promises. While many of the nine problems delineated below are germane to all 
innovations, a few are unique to ^plied Performance Testing. 

1. The Political -Mandate Problem- As previously mentioned, APT is attrac- 
tive to politicians and interest groups who aspouse accountability in education. 
Often these people attempt to have legislation enacted at state level which will 
mandate specific criteria to which schools must conform. Such attempts are well- 
ijitentioned but naive. Communities are quite different and so are schools. To 
establish minimum standards to which all students in a state must conform is both 
mfair and menforceable^ It is mlikely that many students in some inner-city 
schools or those from poor, rural areas will be able to perform the competencies 
that the majority of students in affluent > suburban communities will be able to 
exhibit. 

Conversely, if state standards are predicated on criteria oriented toward 
performances that underpri^/ileged youngsters in poor school districts can exhibit , 
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the standards would be so minimal they will be meaningless. It is difficult 
enough for any one school district to develop standards that will accommodate 
a wide range of student abilities. Furthermore, if APT is mandated but specific 
assessment techniques and methods of evaluation are left to local districts, the 
'•fudge factor*' so prevalent in industrial quality control undertakings will be- 
come rampant. That is, if forced to do so, people will present pretty much what 
the evaluators want to see particularly if they are allowed to assess outcomes in 
their own ways. Applied Performance Testing has a much greater chance of account- 
ing for some of the school's output if it is voluntarily selected, designed, im- 
plemented and assessed at the local level. Yet, it is becoming quite apparent 
that if educators do not develop better evidence of student output, the demand 
for assessment will come from the public and often through legislative channels. 

2. The Overreaction Problem. Educational systems are often slow to respond 
to needs and stress. Then they overreact, overcompensate, and implement poorly 
and fail. The administrator must be wary of overreacting to pressures to imple- 
ment an APT program. Oftentimes nothing is done until social, political and 
economic forces become so great that the system is forced to react. As Anderson 
has indicated, this implementation is atten3)ted with a lack of resources, a short 
time frame, little staff involvement in planning, an inadequate record-keeping or 
management information system, no planning model, little technical development 
and weak assessment techniques, (p, 4) 

3. The Content Problem. Applied Performance Testing is not applicable to 
all content in the curriculum. It is much more relevant for psychomotor learn- 
ings and those in the lower level of the cognitive realm. It is difficult to 
devise performance tests that deal with analysis, synthesis and transfer, although 
some atten3)ts are being made to do this. (See Nickse) Furthermore, as yet there 
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is little understanding of how APT may reflect development in the affective 
domain. Therefore, it should be used in those areas where it works best. Cer- 
tainly we must strive to develop performance tests in the higher levels of cog- 
nitive learning and perhaps even in the affective area. However, one must be 
cautious and not oversell APT's potential with hollow promises that cannot be 
fulfilled. 

4. The Grade Level Problem. Simply stated, at what age or grade level 
should children be measured on which competencies? Obviously, it is important 
for a five-year-old to know how to cross the street and how to interpret basic 
signs. It may even be sensible to have twelve-year-olds in certain rural areas 
be able to drive a tractor. However, is requiring an eighth-grader to fill out 
an income tax form a valid competency? Furthermore, if it is indicated in his 
record that he was able to do this as an eight -grader, what does this mean four 
years later when he graduates from high school? Even if he can remember how to 
do it, the form has probably changed during those years. Thus, it would appear 
that the majority of coupetencies oriented toward adult life should be performed 
in the last two to three years of high school. Other competencies related to the 
immediate life needs of younger children should be taught as they become appropriate 

5. The APT equals Quality Education Problem- Applied Performance Testing 
can be an important output indicator for educational systems. However, it does 
not reflect performance in many learning areas. In fact, it has been called, "A 
twig on the branch of the tree of education." Perhaps this is too much of a re- 
verse hyperbole, yet it should help the administrator keep APT in its proper scope. 

6. The Can-do, Will-do Problem. If a student can demonstrate a particular 
competency— for instance, writing a letter to his congressman or show how to use 
a voting-machine--does this mean he necessarily will do it in the future? With- 
out long-range assessment studies, which have not been possible at this time, this 
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question is impossible to answer. It represents an excellent caution concerning 
the potential neglect of attitudes and values that the thrust toward basic educ- 
tion brings. If we develop only competencies but not the necessary attitude - 
and values that encourage the proper use of these competencies, we have d',nc- 
llttle to improve the quality of life in our communities. 

7. The Economic Resource Problem. In this time of tight budgets and limited 
resources, APT may be an expensive undertaking. A great deal of expertise is 
needed to develop a systemwide model. Staff retraining is required. It is expen- 
sive to develop alternate forms of individualized tests that often require one 
scorer for each individual or small group of individuals. Often alternate sites 
outside the school are needed, and transportation must be made available. The 
writing of objectives, purchase of equipment and development of a fairly complex 
record-keeping system are necessary. It is much better to start with a small 
pilot model that can be adequately svtpported and built on incrementally than to 
attempt implementation of a grandiose undertaking that has little economic support. 

8. The Teacher Involvement Problem. The teaching staff can offer either 
the greatest help or present the greatest barrier to successful implementation of 
any innovation. An application of Gross's statements to APT indicates that the 
following concerns must be attended to. (p. 149) Teachers must clearly understand 
the new role requirements involved in innovations. It appears that APT will force 
teachers to become more oriented toward the communities in which they teach and 
will lower the privacy of the teaching act. This may present a severe threat to 
Some teachers who are accustomed to being lord and master of thier own classroom 
domains . 

Staff capabilities to deal with the innovation must be developed. This may 
be one of the major problems for the administrator in implementing APT. An 
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important competency that needs attention relates to teachers' understanding of 
test theory and development. Not many teachers have an adequate grasp of con- 
cepts related to classical norm-referenced testing theory. Only a few have some 
understanding of criterion referenced and mastery test construction. Applied Per- 
formance Testing draws from both areas of testing theory. Therefore, retraining 
of teachers to enable them to become at least knowledgeable consumers and, hope- 
fully, test developers, would seem to be a high priority consideration for adminis- 
trators considering adoption of APT. 

A third problem cited by staff members as a major reason for innovational 
failure relates to the availability of instructional materials and equipment. At 
present, APT resource centers are beginning to generate support and help in this 
area; some organizations have been hard at woik and have produced materials aimed 
at testing specific competencies. At the ttcsent time, NWREL, HUMRRO, ETS and the 

w 

external high school diploma program in Syracuse, NY, can supply specific sugges- 
tions and materials that may serve as useful exemplars, 

9. The Administrative SiqDport Problem. A final problem area related to 
proper administrative arrangements and support. Time and time again, lack of a 
proper management support system causes innovations to flounder and then fragment 
as teachers cannot find the direction and resources they need. Their motivation 
is lowered, and those who were advocates now become adversaries. The publication. 
An Administrator's Handbook on Applied Performance Testing , addresses itself to 
this concern. 

Once the administrator is aware of the potential problems described above, 
he can begin to develop strategies for dealing with them. These strategies may 
be subsumed under three general categorical areas related to developing support, 
organizing for action and seeking resources. 
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Developing Support for APT 

Tho success of any APT testing program will be heavily dependent on the abil- 
ity of administrators to develop commitoent ar.d provide involvement on the part 
of staff, community, and teachers. If APT is mandated from above without con- 
sidering the needs and ideas of those involved in implementing it at the opera- 
tional level, the chances for its long-term success will be greatly diminished. 
People support undertakings that they believe in, and this validation is primarily 
a result of allowing staff members and citizens to determine major policy and the 
opportunity to participate in specific functional decisions. It is true that 
most people will do what they are ordered to do but often in a mechanical, per- 
functory way that leads only to performance that is minimally acceptable. 

Thus, from the beginning, it is necessary that the administrators involve 
people *flio display interest and a willingness to become involved. All teachers 
might be exposed to an overview of APT in a one- or two-day workshop. Then those 
who feel it has possibilities for their discipline area might receive further in- 
formation and training. 

Public interest might be elicited through newspaper articles, presentations 
to civic organizations, P.T.A. meetings, neighborhood coffees, and personal con- 
tacts. However, many of these contacts should be established with citizens who 
are usually not involved in school -community undertakings. This will require 
initiative and imagination on the part of the administrator, who will have to 
visit actual work locations and speak to many blue-collar workers at union meet- 
ings and socials. This may be a new experience for some vAo are used to dealing 
only with the more prominent citizens in the community. 

One of the major roles of the administrator will be to develop relationships 
between his teaching staff and the community. He v/ill have to remember that 



13 



12 

both groups will be quite anxious in the beginning about sharing their ideas 
with one another. It will be best to start with a small group of the most inter- 
ested citizens and teachers. 

The worst mistake might be to undertaJ:e APT on a large scale before it has 
been carried out as a pilot project- This pilot cycle will allow those involved 
to make suggestions and deal with problems at a manageable level. It will also 
provide an estimation of just what resources might be needed for expansion. It 
is far better to build incrementally on a small project than attempt to implement 
a large-scale testing program without proper resources, training, and knowledge. 
These factors, rather than the validity of the concept, often determine the ulti- 
mate success or failure of educational imdertakings. 

Too often, the following conmients made by staff members when programs are 
in trouble reflect a failure to consider the points made above: 

"We really didn't have any choice , The superintendent and principal had al- 
ready decided we were going to do it J' 

•'It seemed like a good idea when we had the workshop, but we really didn't 
understand how to make it work. We just didn't have enough trainings" 

"A lot of the teachers don't really like it, and so they don't support it when 
they talk with parents and citizens." 

"It's a good idea, but you can't do it unless you have the equipment, materials, 
and help you need to make it work," 

The following summation emphasizes how administrators can avoid the pitfalls 
that generate comments like those above. Expose people to APT; plant the seeds 
for growth. Make the program theirs. Allow those who exhibit a high interest to 
be involved, and let them develop not only specific, operational techniques but 
also general policies and guidelines. Start small; get through one cycle; 
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reexamine and build incrementally on what is good. Be consistently receptive to 
all feedback --even that which is painful. Nurture, develop, and provide oppor- 
tunities for citizens and teachers to work together „ Be a catylist, and stimulate 
action. Provide opportunities for those involved to meet and exchange ideas- 
Remember, support is directly associated with ownership. People will support and 
defend that which they have built. 

Suggested Research 

What do the preceding comments suggest regarding needed research relative to 
administrative concerns as they relate to Applied Performance Testing programs? 
Since it is this author's opinion that the success or failure of such programs 
will result from both the univariate and combined effects of a number of variables 
alluded to in this paper, an examination of these factors will be of eminent im- 
portance- Figure 1 attempts to depict 15 possible correlates. KJbviously, these 
concepts are quite fuzzy. They will have to be operationalized if they are to 
generate quantifiable data--a task of no small magnitude „ It is then with a great 
deal of caution that I offer the following tentative hypotheses: 

1. The higher the degree of administrative commitment the greater the proba- 
bility of perceived* and actual* success of an APT program. 

2. The higher the degree of administrative understanding the greater the proba- 
bility of perceived and actual success of an APT program. 

3. The higher the degree of staff commitment ihe greater the probability of 
perceived and actual success of an APT program 

4. The higher the degree of staff understanding the greater the probability of 
perceived and actual success of an API program, 



^•'Perceived'* success will be measured by solitmg the opinions of administra- 
tors, teachers, students and citizens, ^'Actual" success will be measured by assess- 
ing student performance.. 
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So The greater the degree of crannxinity involvement the greater the proba- 
bility of perceived and actual success of an APT program <> 

6. The smaller the scope of the program during the first cycle of impla- 
ncntation the greater the probability of perceived and actual success of an APT 
program. 

1 . The greater the economic support the greater the probability of perceived 
and actual success of an APT program c 

80 The greater the involvement of specialists from curriculum, management 
and measurement the greater the probability of perceived and actual success of an 
APT program. 

9, The greater the relevance of curriculum content the greater probability 
of perceived and actual succe&« of an APT program, 

10- The more adequate the effort to keep the public informed the greater the 
probability of perceived and actual success cf an APT. program- 

11 « The more adequate the time for development the greater the probability of 
perceived and actual success of an APT program. 

12^ The lower the influence of political mandate (i.e., from the state level) 
the greater probability of perceived and actual success of an APT program, 

13. The higher the involvement of specialists the greater the probability 0I: 
perceived and actual success of «n APT program. 

14 « The more adequate the supportive management system the greater the probability 
of perceived and actual success of an APT program. 

15, The lower the emphasis on the use of APT as a teacher accountability tool 
the greater the probability of perceived and actual success of an APT program. 

Of course, by using multiple regression techniques and path analysis, any com- 
bination of the above variables might be considered. I am sure that others could be 
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added as construct validity for inclusion is developed by examining programs in 
operation. What is of crucial importance is for the administrator to under- 
stand the complexity involved in managing an APT program. Hopefully, this paper 
has helped to clarify the potential and problems inherent in such an undertaking. 
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